Summary

• Adds img2img (image-to-image) generation to lib-infer-diffusion using FLUX in-context conditioning — the reference image is attended to via joint attention, NOT mixed with noise
• Wires the full JS → C++ pipeline: PNG/JPEG dimension auto-detection, Uint8Array serialization, and automatic mode selection (init_image present → img2img, otherwise → txt2img)
• Includes C++ unit tests, JS integration tests, and example scripts

How it works

The user passes init_image (a PNG/JPEG Uint8Array) alongside a text prompt. Internally:

1. The reference image is VAE-encoded into separate latent tokens
2. The target starts from pure noise (not noised input)
3. The FLUX transformer attends to both reference and target tokens via joint attention with distinct RoPE positions
4. The model reasons about reference features (skin tone, structure, facial identity) while generating a new image guided by the prompt
   This approach (matching the Iris C engine) produces significantly better results than traditional img2img (VAE encode → add noise → denoise), which loses identity features at high strength and produces artifacts at low strength.

Changes

JS layer

• addon.js — readImageDimensions() extracts width/height from PNG IHDR or JPEG SOFx headers. runJob() serializes init_image Uint8Array as ref_image_bytes JSON array and auto-injects dimensions to prevent GGML tensor shape assertions.
• index.js — _runInternal() auto-selects mode: img2img when init_image is present, txt2img otherwise.

C++ layer

• SdModel.cpp — load() sets vae_decode_only = false for VAE encoder graph. process() decodes ref_image_bytes, sets ref_images + auto_resize_ref_image for FLUX joint-attention conditioning.
• SdGenHandlers.cpp — Mode handler validates txt2img and img2img.

Tests

• test/unit/test_img2img.cpp (301 lines) — JSON round-trip, dimension override, strength bounds, synthetic image pipeline, cancel
• test/unit/test_ref2img.cpp (390 lines) — reference image routing, auto-resize, full FLUX2 generation with real headshot
• test/integration/generate-image-flux2-i2i.test.js (175 lines) — end-to-end FLUX2-klein img2img

Examples

• examples/img2img-flux2.js — FLUX2-klein Q8 img2img
• examples/img2img-flux2-f16.js — FLUX2-klein F16 variant
• examples/img2img-sdxl.js — SDXL img2img
• examples/ref2img-flux2.js — In-context conditioning example

Usage

const response = await model.run({
  prompt: 'a soccer player version of this photo',
  init_image: fs.readFileSync('headshot.jpg'),
  steps: 15,
  guidance: 9.0
})

Test Plan

Build

• npm run build — native addon builds successfully
• npm run test:cpp:build — C++ test binary compiles

C++ Unit Tests

• npm run test:cpp:run:unit — SdModelTest + SdBackendSelectionTest
• npm run test:cpp:run:loading — SdModelLoadingTest
• npm run test:cpp:run:inference — SdSingleStepInferenceTest
• npm run test:cpp:run:generation — SdFullGenerationTest
• npm run test:cpp:run — all C++ tests (includes img2img + ref2img + cancel + gen_handlers)

JS Integration Tests

• npm run test:integration — all JS integration tests
• Individual: generate-image-flux2-i2i.test.js — FLUX2 img2img end-to-end
• Individual: generate-image-flux2.test.js — FLUX2 txt2img
• Individual: generate-image-sdxl.test.js — SDXL txt2img
• Individual: generate-image-sd3.test.js — SD3 txt2img
• Individual: generate-image.test.js — SD1/SD2 txt2img
• Individual: model-loading.test.js — model load/unload
• Individual: api-behavior.test.js — API behaviour validation

Examples (manual)

• bare examples/img2img-flux2.js — FLUX2 Q8 img2img
• bare examples/img2img-flux2-f16.js — FLUX2 F16 img2img
• bare examples/img2img-sdxl.js — SDXL img2img
• bare examples/ref2img-flux2.js — FLUX2 in-context conditioning
• bare examples/generate-image.js — SD txt2img (regression)
• bare examples/generate-image-sdxl.js — SDXL txt2img (regression)
• bare examples/generate-image-sd3.js — SD3 txt2img (regression)
• bare examples/quickstart.js — quickstart (regression)

Regression

• txt2img workflows produce identical output (no init_image → mode stays txt2img)
• npm run lint — JS lint passes